Mohan et al. Implementation Science 2012, 7:103 
http://wwwJmplementationscience.eom/content/7/1/103 A IMPLEMENTATION SCIENCE 



Implementation 
Science 



RESEARCH Open Access 



Sources of non-compliance with clinical practice 
guidelines in trauma triage: a decision science 
study 

Deepika Mohan 1,2 " Matthew R Rosengart 1,2 , Coreen Farris 3 , Baruch Fischhoff 4 , Derek C Angus 1 
and Amber E Barnato 5 



Abstract 

Background: United States trauma system guidelines specify when to triage patients to specialty centers. 
Nonetheless, many eligible patients are not transferred as per guidelines. One possible reason is emergency 
physician decision-making. The objective of the study was to characterize sensory and decisional determinants of 
emergency physician trauma triage decision-making. 

Methods: We conducted a decision science study using a signal detection theory-informed approach to analyze 
physician responses to a web-based survey of 30 clinical vignettes of trauma cases. We recruited a national 
convenience sample of emergency medicine physicians who worked at hospitals without level l/l I trauma center 
certification. Using trauma triage guidelines as our reference standard, we estimated physicians' perceptual 
sensitivity (ability to discriminate between patients who did and did not meet guidelines for transfer) and decisional 
threshold (tolerance for false positive or false negative decisions). 

Results: We recruited 280 physicians: 210 logged in to the website (response rate 74%) and 168 (80%) completed 
the survey. The regression coefficient on American College of Surgeons - Committee on Trauma (ACS-COT) 
guidelines for transfer (perceptual sensitivity) was 0.77 (p<0.01, 95% CI 0.68 - 0.87) indicating that the probability of 
transfer weakly increased as the ACS-COT guidelines would recommend transfer. The intercept (decision threshold) 
was 1.45 (p<0.01, 95% CI 1.27 - 1.63), indicating that participants had a conservative threshold for transfer, erring on 
the side of not transferring patients. There was significant between-physician variability in perceptual sensitivity and 
decisional thresholds. No physician demographic characteristics correlated with perceptual sensitivity, but men and 
physicians working at non-trauma centers without a trauma-center affiliation had higher decisional thresholds. 

Conclusions: On a case vignette-based questionnaire, both sensory and decisional elements in emergency 
physicians' cognitive processes contributed to the under-triage of trauma patients. 
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Background 

Trauma affects one out of five Americans, requiring the 
expenditure of $400 billion in direct medical costs each 
year. Regionalization— tiered levels of care that distribute 
the sickest patients to the highest-intensity hospitals 
(trauma centers)— reduces mortality and morbidity [1-6]. 
Well-established clinical practice guidelines in trauma 
specify when to triage patients to specialty trauma cen- 
ters [7]. Despite concerted efforts to address known bar- 
riers to compliance (e.g., lack of familiarity with the 
guidelines), between 30% to 70% of all patients who 
meet criteria for transfer to a trauma center remain at 
non- trauma centers (under- triage) [8-11]. At the phy- 
sician level, existing quality improvement efforts have fo- 
cused on improving knowledge, modifying attitudes, and 
removing structural and economic barriers to transfer 
[12,13]. The extent to which cognitive aspects of phy- 
sician decision-making contribute to under-triage is 
unknown. 

Based on cognitive theory for discrimination tasks, in- 
appropriate disposition decisions (transfer/not transfer) 
may result from perceptual sensitivity (the ability to dis- 
criminate between patients who do and do not meet 
clinical practice guidelines for transfer) and/or decisional 
thresholds (the tendency to err on the side of false posi- 
tive or false negative decisions) [14]. These two dimen- 
sions of decision making have different determinants. 
Perceptual sensitivity reflects both physicians' knowledge 
of the clinical practice guidelines as well as intuitive 
judgments (heuristics) about which patients meet those 
guidelines. Decisional thresholds reflect variables such as 
attitudes towards the guidelines, incentives, and 
organizational norms. 

Signal detection theory, a basic decision science method, 
allows the measurement of these two dimensions of deci- 
sion making by parsing performance into sensory and 
decisional components [15]. We used signal detection the- 
ory to analyze emergency physician responses to a case 
vignette-based questionnaire. The objective of this study 
was to assess whether decision making in trauma triage 
primarily reflected physicians' perceptual sensitivity or 
their decisional thresholds. Correctly attributing the 
source of non-compliance with clinical practice guidelines 
would allow us to design more effective quality improve- 
ment interventions. 

Methods 

Triage of trauma patients can occur either in the field or 
after an emergency physician has evaluated the patient 
in the Emergency Department (ED) of a non-trauma 
center. The American College of Surgeons - Committee 
on Trauma (ACS -COT) publishes guidelines specifying 
injuries that warrant transfer to a trauma center. They 
recommend the transfer of all patients with moderate to 



severe injuries, defined as either an injury considered to 
be life- threatening or critical' or an Injury Severity Score 
>15. They recommend that emergency physicians make 
their transfer decision based on information obtained 
from a history, physical exam, and chest and pelvis x-ray. 
We conducted a web-based cross-sectional survey 
among a convenience sample of United States (US) phy- 
sicians who worked in the EDs of non- trauma centers, 
and used responses to clinical vignettes, based on case 
histories, to quantify physicians' perceptual sensitivity 
and decisional threshold for trauma triage. 

Development of questionnaire 
Conceptual model 

Comparing physicians' decisions to ACS -COT guide- 
lines can lead to four possible outcomes as shown in 
Table 1: transferring a patient who meets the guide- 
lines for transfer (true positive or 'hit'); not transfer- 
ring a patient who meets them (false negative or 
'miss'); transferring a patient who does not meet them 
(false positive or 'false alarm'); or not transferring a 
patient who does not meet them (true negative or cor- 
rect rejection') [15]. In contrast to more familiar indi- 
ces of performance that also rely on 2x2 tables for 
exposition, such as sensitivity and specificity, signal de- 
tection theory parses sensory from decisional factors 
influencing performance [14]. As depicted in Figure 1, 
physicians perceive injuries as falling along a con- 
tinuum ranging from very minor to severe. The signal' 
distribution shows their perception of severity for 
patients who should be transferred according to the 
guidelines; the 'noise' distribution shows their percep- 
tion of patients who should not be transferred accord- 
ing to the guidelines. The further apart these 
distributions, the greater their perceptual sensitivity. 
The decisional threshold is the point on the continuum 
above which physicians transfer patients. If perceptual 
sensitivity remains constant, then as the decisional 
threshold moves to the right, physicians tend to err to- 
ward false negative decisions (not transferring patients 
with the 'signal' of serious injury). As it moves toward 
the left, they tend to err toward false positive decisions 
(transferring patients with the 'noise' of less serious in- 
juries) [15]. 



Table 1 2x2 table used to categorize identification 
decisions 





Meets reference 
standard 


Does not meet the 
reference standard 


Decision to 
transfer 


True positive (Hit) 


False positive (False alarm) 


Decision not to 
transfer 


False negative (Miss) 


True negative (Correct reject) 
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Perceptual sensitivity • 




Perceived injury severity 



Figure 1 Signal and noise distribution in theory. 



Clinical vignettes 

Using the ACS -COT guidelines for the transfer of 
trauma patients as our reference standard, we con- 
structed 50 clinical vignettes based on the case histories 
of patients admitted to the University of Pittsburgh 
Medical Center - Presbyterian Hospital trauma service 
[1]. Each vignette included all the information the phys- 
ician would ordinarily obtain from a history, physical 
exam, chest and pelvis x-ray (Le., all the information 
the ACS-COT considers necessary to triage the pa- 
tient). We presented the information in the format of 
a completed trauma care flow sheet, a method recom- 
mended by the ACS -COT to standardize the capture 
of pertinent data. Figure 2 depicts an example of a 
case vignette. By design, one-half the vignettes met 
ACS-COT criteria for transfer (mean Injury Severity 
Score [ISS] 21, range 9-48) and one-half did not 
(mean ISS 2.5, range 1-4), based on independent re- 
view by three trauma surgeons (kappa score for agree- 
ment = 0.85). We systematically varied the complexity 
of the cases to encompass the range of possible triage 
decisions. Thirty-three vignettes described a blunt in- 
jury mechanism and seventeen a penetrating mechan- 
ism. We designed the stimuli so that age, gender, and 
mechanism of injury were unrelated to injury severity 
or need for transfer. 



Clinical vignette pre-testing 

To ensure that our vignettes and procedures were 
clear and without leading information, we conducted 
one round of pretests using cognitive interviews with 
University of Pittsburgh fellows and faculty who were 
board-eligible or certified in EM (n = 9). After revising 
our materials, we conducted additional cognitive inter- 
views with emergency physicians who practiced in 
non-trauma centers in western Pennsylvania (n = 17). 
As a result of this second round of testing, we shifted 



from forced-choice options to free-response text boxes 
to avoid the priming that fixed options suggested. We 
also reduced the set of vignettes to 30, after finding 
that some physicians could not stay fully engaged 
while evaluating all 50. 

Physician sample 

We recruited physicians at a national meeting of the 
American College of Emergency Physicians (ACEP) in 
Fall 2010, using a booth in the Exhibition Hall. We 
designed the study to capture a moderate effect size of 
0.15 for the association between clinical practice guide- 
lines and transfer decisions in a regression model with 
two independent variables (a = 0.05, l-p=0.80) [16]. 
Anticipating a 25% response rate, we recruited 280 phy- 
sicians. Physicians eligible for participation had com- 
pleted residency and cared for adult patients in the ED 
of either non-trauma centers or Level III/IV trauma 
centers in the United States. Those who completed the 
questionnaire received a $100 payment. 

The University of Pittsburgh Institutional Review 
Board study reviewed and approved the study. 

Study protocol 

Participants completed the web-based protocol at their 
convenience. After logging in, they read a brief passage 
describing the task, completed a demographic survey, 
and evaluated the vignettes. 

The demographic survey included questions about 
age, gender, race, educational background (board certifi- 
cation, Advanced Trauma Life Support (ATLS) certifica- 
tion, years since completing residency), and practice 
environment (hospital trauma designation, affiliation 
with a Level I/II trauma center, affiliation with an EM 
residency program, size of the community served). 

The evaluation portion had five blocks of six randomly 
ordered case vignettes, with each block containing two 
cases with moderate-severe injury severity/blunt mech- 
anism, one with moderate-severe injury/penetrating 
mechanism, two with minor injury/blunt mechanism, 
and one with minor injury/penetrating mechanism. Phy- 
sicians used a free-response text box to answer the ques- 
tion, 'What would you do to manage the patient?' The 
instructions asked them to provide information about 
treatment, interventions, and disposition. They had two 
weeks to complete the survey. 

Analysis 

Descriptive summary of responses 

We coded each free-text response to identify manage- 
ment decisions related to: disposition, consults, im- 
aging, medications, laboratory studies, procedures, and 
resuscitation efforts. To assess inter-rater reliability, 
two coders evaluated a random sample of 10% of the 
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TRAUMA CARE FLOW RECORD 



Assumptions 

• Patient on back board with c-collar in place. 
•2186 IVs in place. 

• I VF hanging. 

• Exam normal except where noted on primary and secondary survey. 



Mechanism of Injury: 
■ MVC 

□ A TV 

□ GSW 



□ Stab 

□ Assault 
D Motorcycle 



SECONDARY SURVEY 



□ Pall 

□ Other 



Chief Complaint: 

44 year old male, motor vehicle collision, collision with utility pole. 
Unrestrained driver (♦JLOC 



Steering wheel impression 
on chest and upper 
abdomen 



Tender to palpation 
— overt chest 



PRIMARY SURVEY 



Respiratory 

Breath Sounds R 

Clear ■ □ 

Dimished O H 

Absent □ □ 



Airway 



■ Patent 
O Compromised 
□ Obstructed 



Cardiovascular: Pulse 

H Strong 

□ Palpable 

□ Weak 



Vitals: Pulse 100 

Rhythm Regular 



BP 170/80 
Pulse OX 92% 



RR 20 
Oz 4L 




Tender to palpation 
over L& Rankle 



GLASGOW COMA SCALE: 



GCS TOTAL: 14 



Eye Opening 

Spontaneous 



Verbal Response 



Motor Response 



Medical History: 



Morbid obesity (300lbs), diabetes. 



Oriented 



Obeys Commands 



hypertension, 



To Voice 



Confused 



Localizes (Pain) 



To Pain 



Inapprop. Words 



Withdraws (Pain) 



Medications: 



lismopril, lopressor, metformin 



None 



Incomprehensible 2 Flexion (Pain) 



None 



Extension (Pain) 



None 



Neurological: 

Orie ntedto: 
H Person 
| Time 

Baseline: GCS 15 



Pupils: 

□ Place 

PI Event Brisk 

Sluggish 
Non-Reactive 



□ □ 

□ □ 



IMAGING 

CXR: Poor quality- possible Pelvis: Poor quality 
widened mediastinum 
and L hemothorax Other: 



Figure 2 Example of case vignette. 



management decisions. We found very high agreement 
for all six types of management decisions (kappa 
0.81-0.94) [17]. We used logistic regression models, 
clustered by physicians, to estimate the predicted 
probability that physicians would make types of man- 
agement decisions for minor and moderate to severe 
cases. 

We then scored each response dichotomously as com- 
pliant or non-compliant with ACS-COT guidelines. For 
patients with moderate to severe injuries, we defined 
compliance as: transferring the patient to a trauma 
center without obtaining additional testing; transferring 
the patient after performing additional testing contin- 
gent on not delaying the transfer; or consulting trauma 
surgery (assuming that the surgeon would ensure the 
appropriate triage). For patients with minor injuries, 
we defined compliance as: not transferring the patient 
to a trauma center; transferring a patient to a trauma 



center contingent on identifying further injuries; or 
consulting trauma surgery. 

Physician performance 

We analyzed performance in two ways. First, we calcu- 
lated adherence to standard ACS -COT triage bench- 
marks. The ACS -COT defines under-triage as the 
proportion of patients with moderate-severe injuries 
who are not transferred. It defines over-triage as the pro- 
portion of transferred patients who have minor injuries 
[7]. We summarized rates of under- and over- triage for 
each physician. 

Second, we used a regression-based approach to signal 
detection theory to estimate physicians' perceptual sensi- 
tivity and decisional threshold. With adequate trials 
(>100) [18], one can estimate an individuals perceptual 
sensitivity and decisional threshold directly from their 
decisions. Where the cognitive burden of a task limits 
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the number of trials, as it did here, one can use a 
regression-based approach to estimate the parameters 
and error estimates [19]. 

Perceptual Sensitivity (d) 

= logit(Hit Rate)-logit(False Alarm Rate) 
= ln(Hit Rate)- ln(Miss Rate) 
- ln(False Alarm Rate) + ln(Reject Rate) (1) 

Decisional Threshold (c) = — logit (False Alarm Rate) 
= — ln(False Alarm Rate) + ln(Reject Rate) 

(2) 

When these two equations are combined, they form 
the logistic regression model: 

logit p(Y = 1|X) = -c + dX (3) 

where Y is the disposition decision and X is the refer- 
ence standard [20]. The fit of logistic models approxi- 
mates signal detection models, but with a scale n/V3 
times larger than the Gaussian distribution used in sig- 
nal detection theory [20]. 

We fit a model for each physician, predicting the log- 
odds of his or her disposition decisions (dependent vari- 
able) using an intercept and a regression weight on the 
ACS-COT guidelines for transfer (independent variable). 
The models intercept predicts disposition decisions in 
the absence of any information, hence provides an esti- 
mate for the decisional threshold [20]. The regression 
weight on the ACS-COT guideline variable represents 
perceptual sensitivity, showing the degree of reliance on 
guidelines (implicitly or explicitly) when making dispos- 
ition decisions [20]. We scaled our estimates by 1.8 to 
approximate the standard deviation units of signal detec- 
tion values. 

A perceptual sensitivity estimate greater than zero 
indicated a positive association between the guidelines 
and decisions to transfer patients. A negative decisional 
threshold suggested a tendency to transfer trauma 
patients (i.e., effectively preferring false positives to false 
negatives), and a positive value suggested a tendency not 
to transfer. As accuracy approached the limit (15 of 15 
true positive or true negative cases transferred appropri- 
ately), the estimation procedure failed. We corrected for 
perfect scores by converting proportions of 0 and 1 to 1/2N 
and 1-1/2N (where N equaled the total number of trials) re- 
spectively [21]. 

Applying the ACS -COT benchmarks for triage 
resulted in the following relationships. 



Physicians who achieved <5% under- triage had: 

Perceptual Sensitivity > logit (0.95) 

+ Decisional Threshold 

(4) 

Physicians who achieved <50% over-triage had: 

Perceptual Sensitivity > 0 (5) 

In subsequent analyses, we used a random effects lo- 
gistic regression model to estimate the perceptual sensi- 
tivity and decisional threshold for the entire cohort, 
adding mechanism of injury (blunt versus penetrating) 
as an additional predictor of physician disposition deci- 
sions. We also used Student's t-test and Spearman corre- 
lations as appropriate to estimate the influence of 
physician characteristics (age, gender, race, board certifi- 
cation, ATLS certification, years since completing resi- 
dency), and practice environment (hospital trauma 
designation, affiliation with a Level I/II trauma center, 
affiliation with an EM residency program, size of the 
community served) on perceptual sensitivity and decisio- 
nal thresholds, and tested these characteristics in a mul- 
tivariable linear regression model. 

Results 

Physician sample 

Participant enrollment and response rates are shown in 
Figure 3. Among the 280 physicians recruited at ACEP, 
210 logged on to the website (75%), of whom 202 met 
eligibility after reviewing their educational and practice 
background information. Of the 202 eligible physicians, 
168 (80%) completed the questionnaire. The mean age 
of the physicians who completed the survey was 41.5 
years (SD = 9.55). 157 (93%) were board-certified in 



280 Physicians recruited at national meeting 
of American College of Emergency 
Physicians 



210 Physicians responded (75%) 



Physicians ineligible: 

• 4 worked at Level l/ll TC 

• 3 practiced pediatric EM 

• 1 practiced outside the US 



Incomplete responses: 

• 13 attempted to complete 
•> after survey deadline 

• 5 task too onerous 

• 1 admitted to hospital 

• 15 unknown 



168 Physicians completed the survey (80%) 



Figure 3 Physician subject enrollment and participation rates. 

V ) 
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Emergency Medicine and 125 (74%) certified in 
Advanced Trauma Life Support. 48 (29%) worked in a 
hospital affiliated with a level I/II trauma center. Partici- 
pant characteristics are shown in Table 2. 

Analysis 

Descriptive summary of responses 

On average, physicians spent 2.3 minutes (SD = 2.3) per 
vignette in reading and responding, and listed 2.9 



Table 2 Demographic characteristics of physicians 



Variable 


Value 


Age, mean (SD) 


41.5 (9.55) 


Gender, n(%) 


Male 


141 (84) 


Female 


27(16) 


Race, n(%) 


Caucasian or white 


127 (76) 


African American or black 


8(5) 


Hispanic or Latino 


9(5) 


Asian 


20 (12) 


Native American 


1 (0.6) 


Pacific Islander 


2(1.2) 


Other/undocumented 


1 (0.6) 


Primary specialty, n(%) 


Emergency Medicine 


157 (93) 


Family Practice 


7(4) 


Internal Medicine 


3(1) 


Other 


1 (D 


Years since completing residency, mean (SD) 


9 (9.25) 


ATLS certified, n(%) 


Yes 


125 (74) 


No 


43 (26) 


Years since ATLS certification, mean (SD) 


3 (6.6) 


Designation of hospital where 




physician works, n(%) 




III 31 (18) 


IV 


4(2) 


Non-trauma center (non-TC) 


133 (79) 


Affiliation of the hospital with a TC 




(if working at a non-TC), n(%) 




Yes 


48 (29) 


Affiliation of the hospital with an 




EM residency program, n(%) 




Yes 


26(15) 


Size of community served by hospital, n(%) 


>1 million 


19 (11) 


250,000 - 1 million 


54 (32) 


200,000 - 249,999 


76 (45) 


2,500 - 199,999 


18(11) 


<2,500 


1 (0.6) 



decisions (SD = 1.1; range 1-6). For example, one vi- 
gnette involved a patient sustaining a fall from a 15 -foot 
balcony, who presented with seizure-like activity. We 
had categorized the patient as having a moderate to se- 
vere injury, based on the ACS-COT guidelines. Partici- 
pant 110 described this management strategy: order 
anti-epileptic medications, perform an intubation, obtain 
a CT of the head and C-spine, and transfer the patient 
to a trauma center. We coded that response as not com- 
pliant with the guidelines because he/she delayed trans- 
fer to obtain a CT scan. A summary of types of decisions 
made for minor and moderate to severe cases is pre- 
sented in Table 3. 



Physician performance 

The median under-triage rate among physicians was 
80% (Inter-Quartile Range [IQR] 40-97%). In other 
words, physicians expeditiously transferred 20% of 
patients meeting the guidelines for transfer. The most 
common reason for under-triage was delaying transfer 
to obtain additional imaging. The median over-triage 
rate was 29% (IQR 0-47%). In other words, 71% of 
patients transferred met the reference standard. Sixteen 
participants (9%) did not transfer any patients, but con- 
sulted specialists at their own institutions for further 
management of all cases. 

In the logistic model for the entire cohort, the regres- 
sion coefficient on ACS-COT guidelines for transfer 
(perceptual sensitivity) was 0.77 (p<0.01, 95% CI 0.68- 
0.87) indicating that, on average, the probability of trans- 
fer weakly increased as the ACS -COT guidelines would 
recommend transfer. The intercept (decision threshold) 
was 1.45 (p<0.01, 95% CI 1.27-1.63) indicating that, on 
average, participants had a conservative threshold for 
transfer, erring on the side of not transferring patients. 
Estimates of physician perceptual sensitivity and decisio- 
nal threshold are graphically presented in Figure 4. 

As shown in Figure 5, there was significant between- 
physician variability in perceptual sensitivity and decisio- 
nal thresholds. 72 (43%) physicians had a perceptual 
sensitivity estimate greater than 1, indicating better than 
moderate ability to discriminate between patients who 
did and did not meet guidelines for transfer. 75 (45%) 
physicians had a decisional threshold estimate less than 
1.45, demonstrating a lower tolerance for errors on the 
side of not transferring patients. The ACS -COT recom- 
mends that physicians at non-trauma centers transfer 
95% of patients with moderate-severe injuries, even if 
that means that up to 50% of all transfers have minor in- 
juries [7]. Given the limited number of cases on our 
questionnaire, meeting these benchmarks required per- 
fect ability to identify patients who met the guidelines 
for transfer. Only two physicians (1%) did so. 
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Table 3 Predicted probability of management decisions made for patients with minor and moderate to severe injuries 



Decisions 


Patients with minor injuries 




Patients with moderate-severe 


injuries 




Not transferred 


Transferred 


Not transferred 


Transferred 


Disposition 


37% 




54% 




Consults 


23% 


5% 


36% 


23% 


Imaging 


91% 


46% 


88% 


38% 


Medications 


38% 


35% 


33% 


30% 


Labs 


33% 


21% 


29% 


18% 


Procedures 


32% 


41% 


57% 


66% 


Resuscitation 


7% 


15% 


48% 


28% 



Including mechanism of injury in the analysis revealed 
that cases with penetrating (versus blunt) injuries were 
more likely to be transferred (OR 173, p<0.01, 95% CI 
1.46-2.03), even though they were no more (or less) 
likely to meet ACS -COT guidelines in the sample 
vignettes. 

Physician characteristics {e.g., gender, training) did not 
influence perceptual sensitivity. Men had higher decisio- 
nal thresholds than women (p = 0.01). Physicians with 
board certification in Emergency Medicine compared to 
physicians with board certification in Internal Medicine 
or Family Practice (p = 0.03), and physicians working at 
non-trauma centers without an established affiliation 
with a trauma center compared to physicians working at 
non-trauma centers with an affiliation (p<0.01) had 
higher decisional thresholds. Two of these associations 
(gender and affiliation of their hospital with a trauma 
center) remained significant in the multivariable linear 
regression model. 

Discussion 

In a vignette -based study with 168 emergency physi- 
cians, we used signal detection theory to quantify the 
perceptual sensitivity and decisional threshold of physi- 
cians making trauma triage decisions. We found that 




Perceived Injury Severity 



Figure 4 Signal and noise distribution in practice. 

V / 



cognitive processes in physician decision-making may 
contribute to persistent rates of under-triage in trauma 
triage. When responding to the cases, many physicians 
in the study had low perceptual sensitivity, making triage 
decisions only weakly related (if at all) to the ACS -COT 
standard. Additionally, most had a high decisional 
threshold for transfer, systematically erring on the side 
of not transferring patients to regional trauma centers. 

Many epidemiological studies have shown that patients 
with moderate to severe injuries are routinely under- 
triaged [8-11]. However, those studies have typically 
focused on patient-level determinants of variability in 
triage decisions. Chang et al. have described age as a 
determinant of triage decisions [10]. Macias et al. posit 
that age, co-morbidities, and severity of injury influence 
the triage of patients with traumatic spinal cord injuries 
to trauma centers [8]. We used signal detection theory, a 
basic decision science method, to determine how phys- 
ician cognitive processes might influence rates of under- 
triage. Although rarely used for this purpose, signal 
detection theory has the advantage that it distinguishes 
between sensory and decisional components of decision 




-10 12 3 

Decisional Threshold 

Figure 5 Individual physicians' decisional thresholds and 

perceptual sensitivity. The dotted grey lines indicate the decisional 

threshold and perceptual sensitivity above which physicians would 

meet the ACS-COT benchmarks for triage. 
\ . ) 
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making. For example, by using signal detection theory to 
analyze nursing risk assessments for patients admitted to 
hospital, Thompson et al. have shown that time-pres- 
sure and clinical experience influence variability in deci- 
sion making through different mechanisms. We 
hypothesized that signal detection theory would allow us 
to identify reasons for non-compliance with clinical 
practice guidelines. Better understanding of the cognitive 
processes responsible for under-triage would provide 
valuable information for the design of future quality im- 
provement interventions [22]. 

We presented physicians with a series of case vignettes 
that required a range of triage decision-making. We 
found that when answering the case vignettes most phy- 
sicians demonstrated limited perceptual sensitivity, as 
shown in Figure 4. The ACS-COT uses ATLS, an educa- 
tional program that operationalizes the clinical guide- 
lines, as one of its primary tools to standardize the 
treatment of trauma patients. However, although most 
physicians in our sample had received ATLS certifica- 
tion, the regression weight on the guidelines in our 
models suggested that triage decisions on this question- 
naire corresponded only weakly with the ACS -COT cri- 
teria for the transfer of patients. 

Patterns in the responses suggested potential explana- 
tions for the limited perceptual sensitivity demonstrated 
by these physicians. For example, as shown in Table 3, 
among one-half of the under-triaged patients, physicians 
correctly identified the patient as meeting the standard 
for transfer. However, they delayed transport to obtain 
additional diagnostic imaging. Acquisition of this im- 
aging suggested triage decision-making consistent with 
the disjunction effect. As described by Shafir and 
Tversky, people have difficulty making decisions in the 
context of complex uncertainty. One manifestation of 
that difficulty is the pursuit of non-instrumental infor- 
mation, which appears relevant but if available would 
not impact decision making [23,24]. In effect, the infor- 
mation from imaging acquired for patients with moder- 
ate to severe injuries would have had negative value, as 
it reduced the chances of successful treatment. 

Similarly, the greater likelihood of transferring patients 
with penetrating (rather than blunt) injury suggested re- 
liance on the representativeness heuristic. Formally 
equivalent problems should provoke equivalent judg- 
ments. In other words, people should estimate the prob- 
ability of x belonging to set A in the same way that they 
estimate the probability of y belonging to set A. How- 
ever, Kahneman and Tversky have shown that when 
people rely on the representativeness heuristic to make 
judgments, they substitute the perceived similarity of x 
or y to other objects in set A for that probability esti- 
mate [25]. In this context, cases with blunt injuries 
seemed less likely to need tertiary center care as 



otherwise equivalent cases with penetrating injuries. 
This hypothesis would also explain patterns of patient- 
level variability in triage. For example, if physicians judge 
severe trauma as the purview of young men, they may 
systematically under-triage elderly patients, as described 
by Chang et al. and Macias et al. [8,10]. 

Figure 4 also demonstrates that the group of physi- 
cians who responded to the questionnaire had a high 
threshold for transfer. To target physicians' decisional 
thresholds, the ACS -COT has relied on trauma systems, 
voluntary networks of local community hospitals that as- 
sociate with high- volume trauma centers [13]. Through 
outreach and accreditation programs, it has urged physi- 
cians at non-trauma centers to have a low decisional 
threshold in order to minimize under-triage. In other 
words, it advocates transferring as many patients as ne- 
cessary to ensure the capture of all those with moderate 
to severe injuries, even at the cost of increasing over- 
triage [7]. Despite these efforts, our sample of physicians 
preferred to err on the side of minimizing over-triage. 
Free-text comments by participants suggested that their 
decisional thresholds reflected conscious negative atti- 
tudes towards transferring patients, invoking issues such 
as resentment towards guidelines in general and distaste 
at relinquishing control over patient care. The finding of 
lower decisional thresholds among physicians at hospi- 
tals with an established trauma center affiliation sug- 
gested that organizational norms or incentives may play 
roles as well. For example, hospitals without these affilia- 
tions may encourage their physicians to avoid transfer- 
ring patients out of network to prevent the loss of 
revenue. 

Current quality improvement efforts in trauma as- 
sume that the same barriers to compliance with clinical 
practice guidelines affect all physicians equally. How- 
ever, the individual performance differences revealed in 
Figure 5 suggest the need for a more nuanced ap- 
proach. For example, some physicians had a high 
threshold for transferring patients, and an above aver- 
age ability to discriminate between patients with minor 
and moderate-severe injuries. These doctors seemed to 
at least partially compensate for their apparent unwill- 
ingness to transfer patients through skill at distinguish- 
ing between those who really did 'need' transfer and 
those who did not. Other physicians had a threshold 
for transferring patients set around zero and a lower 
than average perceptual sensitivity. These doctors 
seemed to at least partially compensate for their appar- 
ent inability to distinguish between those who did and 
did not 'need' transfer by being more willing to transfer 
everyone. 

Physicians with decisional thresholds biased away from 
transfer and with above-average perceptual sensitivity 
would most likely benefit from an intervention that 
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recalibrated their decisional threshold, perhaps by 
addressing their concerns about transferring patients or 
creating financial incentives for the appropriate transfer 
of patients. In contrast, exposing the second type of 
physician with a decisional threshold already set around 
zero to that same intervention might have unwanted 
consequences. Specifically, increasing their willingness to 
transfer patients would increase over-triage, and could 
impose a burden on level I centers. An analysis of triage 
patterns in Pennsylvania by Mohan et al demonstrates 
that simply shifting decisional thresholds to achieve 
ACS-COT targets for triage would result in a five-fold 
increase in transfers to trauma centers [11]. Moreover, 
non-trauma centers would lose an important source of 
revenue and the opportunity to provide care for patients 
in their community. Instead, these physicians might 
benefit from a strategy that modified their heuristics, 
perhaps through training with stimuli like the vignettes 
used in this study [26] . 

Our study had several limitations. First, we used vign- 
ettes to measure physicians' perceptual sensitivity and 
decisional threshold, which did not replicate the time 
or organizational pressures of clinical decision-making, 
and with less information than may be available in real- 
life. Evidence suggesting that case vignettes can predict 
physician practice patterns comes primarily from the 
outpatient setting. For example, Peabody et al demon- 
strated that physicians' management of paper patients' 
with common clinical conditions, like lower back pain, 
corresponded to their management of real patients 
[27,28]. Our specific vignettes have the content validity 
that comes from basing them on the case histories of 
patients treated at the University of Pittsburgh Medical 
Center. Additionally, the vignettes were extensively 
pretested. However, we have no knowledge of 
whether physician decision-making in response to a 
case vignette -based questionnaire corresponds to ac- 
tual practice patterns. An alternative approach, and a 
possibility for future research, might be using an ad- 
ministrative dataset to calculate physicians' perceptual 
sensitivity and decisional threshold based on actual 
triage decisions. 

Second, we used the ACS -COT guidelines as our refer- 
ence standard for transfer, despite flaws that might affect 
acceptance by physicians: all the stakeholders did not 
participate in their design [29]; the definition of over- 
triage creates an association between prevalence of in- 
jury and performance [29], and the benchmarks may 
lack feasibility [11]. Yet, robust observational data sup- 
ports their overall validity. MacKenzie et al among 
others, have shown that patients meeting ACS-COT cri- 
teria for transfer have better outcomes when treated at 
trauma centers [1]. Moreover, their widespread dissem- 
ination makes them the de facto standard for decision 



making among physicians, regardless of their potential 
failings. 

Third, we used a non-representative sample of cases 
on our questionnaire to allow the use of signal detection 
theory. We did systematically vary the complexity of the 
cases to elicit the range of decisions that physicians 
would perform in practice. However, we used a much 
higher proportion of cases with moderate to severe in- 
juries than physicians would see in practice. We specu- 
late that the absence of non-trauma cases made the 
triage task easier than the one faced by physicians in 
their EDs. However, the higher base rate of severe in- 
juries may have altered customary practice patterns. 
Birnbaum has shown that the predicted effect of the base 
rate on signal detection estimates depends on the theory 
of judgment used [30]. We therefore have no clear feeling 
for how this bias would have affected the patterns 
observed here. As a test of possible learning effects and 
base rate influence on the parameter estimates, we com- 
pared performance on the first half of the study relative 
to the second half and found no differences. 

Finally, we recruited physicians at a national meeting 
of emergency medicine physicians, which might limit 
the generalizability of our observations. However, be- 
cause physicians attending academic meetings likely 
have greater knowledge of current clinical practice 
guidelines, we assume that any bias introduced by our 
sampling frame would be in the direction of better 
practice. 

Conclusions 

Under-triage persists despite comprehensive efforts 
by governmental and professional organizations to stand- 
ardize trauma practice. In responses to a case vignette- 
based questionnaire, we found that under-triage can 
result from cognitive processes used in physician decision- 
making. Further research is required to determine how 
cognitive aspects of physician decision-making affects the 
triage of patients in real practice, as well as how best to 
intervene. 
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